Gu Test: A Progressive Measurement Of Generic 
Artificial Intelligence 


Author: Lifan Gu 


The recent accidents of automated-driving cars (some are fatal) raise some 
serious issues in artificial intelligence (AI) theories and testing, which have 
been ignored for a long time. 


The promoted driverless cars without specific constrains mean SAE level 5 
automated driving, which is impossible due to lack of human level 
intelligence. 


To test human level intelligence, Turing Test is actually invalid, not only it is 
subjective, but also the language complexity is much less than the human 
intelligence complexity. So instead, I design Gu Test, a progressive 
measurement of generic artificial intelligence by their falsifiability. 


1. AI Problems in Reality 


According to news, in 2015 a blind man was allowed to take a driverless car 
alone, before the accident on 02/14/2016 [1]. Although the damage of this 
accident is minor, wrong judgement of driverless cars is very dangerous 
potentially. 


Scientific experiments should be done under strictly controlled conditions. 
Conclusions can only be derived based on conditions. Simulations and 
empirical tests could being misleading. 


According to the Waymo Safety Report 2017, the driverless car is highly 
wired with pre-information [2], making it very difficult to adapt to future 
mode evolution, especially when the mode evolution is not stable. The 
probability of mistakes in future could change significantly with mode 
evolution. Actually nothing in this safety report could test level-5 driverless 
cars. 


If the mode evolution is not stable, not only the judgement based on 
intuition could be wrong, the statistical results, the deep-learning of 
empirical data, and other AI technologies also could be severely misleading. 


The AlphaGo Zero paper on Nature presented a method learning from 
scratch. It claimed a superhuman performance [3]. However the paper did 
not provide any evidence for this claim. 


Superhuman is a concept related to generic human. AlphaGo Zero defeating 
AlphoGo Master is not an evidence of superhuman. AlphoGo Master 
defeating some humans in some games is not an evidence of superhuman, 
either. 


Because the concept of superhuman relates to generic human, it gives a 
wrong impression that these AI technologies have already exceeded generic 
human intelligence [4]. 


So I designed a scientific experiment with strictly controlled conditions to 
falsify this claim or any such implications. 


2. Fundamental Problems in Theories 


The 3rd edition of AI textbook Artificial Intelligence: A Modern Approach 
did not address the test problems properly. This is because it took Aristotle 
philosophy as the foundation by mistake, and even deleted the introduction 
of Socrates and Plato philosophies. 


The textbook did not understand that Galileo set Socratic method and 
experiment as the foundation of sciences in his article Dialogue Concerning 
the Two Chief World Systems. Aristotle philosophy actually is wrong. 


Since this textbook has been used in mainstream societies, after the release 
of this edition, together with the Technological Singularity propagation, AI 
promotion became wild and unrealistic. Now several fatal accidents of 
automated-driving cars happened. 


This edition of the textbook claimed: "the quest for ‘artificial flight’ 
succeeded when the Wright brothers and others stopped imitating birds and 
started using wind tunnels and learning about aerodynamics." 


What important here is not “imitating birds" or not, but to understand the 
principles of forces. Airplanes were not even designed to replace birds 
originally. However, automated-driving (especially SAE level 5) is to replace 
human drivers. So we do need to understand the principles of human specific 
intelligence. Gu Test could verify that we do not yet. 


| do not propose to imitate humans. However, to understand the principles of 
human specific intelligence, we really need to study humans. 


Actually it is “one hundred years before the Wright brothers flew their 
glider, Cayley had established the basic principles and configuration of the 
modern airplane, complete with fixed wings, fuselage, and a tail unit with 
elevators and rudder, and had constructed a series of models to 
demonstrate his ideas.", and Wright brothers acknowledged this [5]. 


Without the previous work from Issac Newton (1643-1727), George Cayley 
(1773-1857) and others, Wright brothers would not make miracles with 
wind tunnel. Sciences and scientific experiments are far beyond empirical 
experiments [6]. 


The problems in this edition of the textbook is not only limited to the lack of 
knowledges in history and philosophy. It did not realize the fundamental 
problems in mathematics and Turing Machine. 


Computers, including quantum computers, have systematic problems to 
process high-order logic and recognize sophism, which are critical to 
sciences and other academic researches. 


Godel theorems suggest mathematics cannot be used to judge the 
correctness of sciences. There is limitation of Turing Machine. So universal 
approximation does not exist on Turing Machine, and Technological 
Singularity is baseless. 


The progresses made from neurosciences are mainly at physiological level 
or animal level, such as in motion, vision, audio, emotion, etc. Nothing could 
illustrate the human specific intelligence so far [6]. 


So a theoretic and systematic study of human intelligence is needed, to 
cross languages, philosophies, mathematics, and sciences. etc. 


3. Gu Test 


Based on my studies, I design some progressive measurement of human 
intelligence based on falsifiability, to test AI systems, and also to test my 
theories [7]: 


1. A 4-dimension experiment space for the strongest Computer Go system, 
to test its intelligence on Go games, especially to test AlphaGo Zero's 


superhuman claim or any such implications due to the problems in deep- 
learning, reinforcement, etc. 


Since there is only one opportunity to gather certain experiment results 
before the computer Go system could be adjusted by humans, the first 
round experiment should be done on the strongest Computer Go system. 


2. A progressive test scheme for natural language processing (NLP) system, 
besides the processing of high-order logic and the recognition of sophism, 
etc., which are necessary to scientific progresses. 


4. Future 


The studies could be extended to other kinds of AI systems, and other 
aspects of human intelligence in future. 


However, life-threatening situations happened to me again and again. I 
cannot do further researches unless in safe personally and economically. 
Actually, some health degrading could be irreversible, so I may not be able 
to do other researches except for the experiment schemes already designed. 
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[4] Such cautiousness is necessary considering that Deepmind was sold to 
Google at a big amount of money, and Google employees did propagate for 
Technological Singularity and driverless cars. 


My experiment scheme for Go game introduced later in this article is to 


verify that AlphaGo Zero does not have generic human intelligence even in 
Go game. Since Deepmind is unwilling to do these experiments, people 
should be even more cautious. 


Navigation is actually an animal level intelligence, as well as vision, audio, 
motion, emotion, etc. So the usage of superhuman for grid-like agents could 
be even more misleading (as 8pm, 05/12/2018, superhuman usage appeared 


at https://deepmind.com/blog/grid-cells/). 


Car-driving is not just navigation. I choose Go gaming, languages, and life 
systems as the focuses of my intelligence studies. 


[5] http://www.ctie.monash.edu.au/hargrave/cayley. html 


"He was the first to identify the four aerodynamic forces of flighttweight, 
lift, drag, and thrustuand their relationship and also the first to build a 
successful human-carrying glider. 


Cayley described many of the concepts and elements of the modern airplane 
and was the first to understand and explain in engineering terms the 
concepts of lift and thrust. Before him, researchers thought that the 
propulsion system should generate both lift and forward motion at the same 
time, as birds were able to do. So they constructed their flying machines 
with flapping wings (called ornithopters) to resemble the motion of birds. 
Cayley realized that the propulsion system should generate thrust but that 
the wings should be shaped so as to create lift." 


"Indeed, in 1909 Wilbur Wright himself paid Cayley the following tribute: 
‘About 100 years ago, an Englishman, Sir George Cayley, carried the 
science of flight to a point which it had never reached before and which it 
scarcely reached again during the last century." 


[6] In the studies of human specific intelligence, so far nothing comparable 
with the physics in Newton age, we are still waiting for 'Copernicus' and 
‘Galileo’ to come. 


Before Scientific Revolution, even without the corret knowledge of physics, 
humans did make some machines based on experiences and geometry, just 
with very limited functionalities. 


[7] Gu Test does not intend to distinguish humans from humans. It only 
measures the difference between generic human and machines, or between 
generic human and other animals. 


